Analyzing the Conceptual Relationship Between ISO/IEC 15504 (Software Process Assessment) and the Capability Maturity Model for Software

The Capability Maturity Model for Software (Software CMM ) is probably the best known and most widely used model world-wide for software process improvement. ISO/IEC 15504 is a suite of standards currently under development for software process assessment, which can be expected to affect the continuing evolution of the Software CMM. This paper discusses the similarities and differences between the two models and how they may influence each other as they both continue to evolve. Introduction The Software Engineering Institute (SEI) has been evolving a process maturity framework now known as the Capability Maturity Model for Software (Software CMM) since 1986 [Paulk95a, Paulk95c]. This model provides organizations with guidance for measuring software process maturity and establishing process improvement programs. The Software CMM is probably the best known and most widely used model world-wide for software process improvement at this writing. ISO/IEC1 15504 is a suite of standards for software process assessment currently under development as an international standard . ISO/IEC 15504 has been published as a type 2 technical report, which is a stage in the development of a standard. Of the nine parts to ISO/IEC 15504, the parts directly relevant to the Software CMM are ISO/IEC 15504-2, the reference model, and ISO/IEC 15504-5, which provides an example model. As an international standard, ISO/IEC 15504 can be expected to affect the continuing evolution of CMM-related products.  Capability Maturity Model and CMM are registered in the U.S. Patent and Trademark Office. The Software Engineering Institute is a federally funded research and development center sponsored by the U.S. Department of Defense. 1 ISO/IEC 15504 is being developed by working group 10 (WG10) under the software engineering subcommittee (SC7) of the information technology joint committee (JTC1) established by the International Organization for Standardization (ISO) and the International Electrotechnical Commission (IEC). 1999 International Conference on Software Quality Cambridge, MA 2 This paper provides an overview of the Software CMM and ISO/IEC 15504, discusses the similarities and differences between them, and speculates how they may influence each other as they both continue to evolve. The Capability Maturity Model for Software The Capability Maturity Model for Software describes the principles and practices underlying software process maturity and is intended to help software organizations improve the maturity of their software processes in terms of an evolutionary path from ad hoc, chaotic processes to mature, disciplined software processes. The Software CMM is organized into five maturity levels, described in Table 1. Table 1. Software CMM Maturity Levels. Software CMM Maturity Level Description of Software CMM Maturity Levels 1) Initial The software process is characterized as ad hoc, and occasionally even chaotic. Few processes are defined, and success depends on individual effort and heroics. 2) Repeatable Basic project management processes are established to track cost, schedule, and functionality. The necessary process discipline is in place to repeat earlier successes on projects with similar applications. 3) Defined The software process for both management and engineering activities is documented, standardized, and integrated into a standard software process for the organization. All projects use an approved, tailored version of the organization's standard software process for developing and maintaining software. 4) Managed Detailed measures of the software process and product quality are collected. Both the software process and products are quantitatively understood and controlled. 5) Optimizing Continuous process improvement is enabled by quantitative feedback from the process and from piloting innovative ideas and technologies. Except for Level 1, each maturity level is decomposed into several key process areas that indicate the areas an organization should focus on to improve its software process. The key process areas in Version 1.1 of the Software CMM are listed in Table 2. For convenience, the key process areas are internally organized by common features. The common features are attributes that indicate whether the implementation and institutionalization of a key process area is effective, repeatable, and lasting. The five common features are Commitment to Perform, Ability to Perform, Activities Performed, Measurement and Analysis, and Verifying Implementation. General practices that apply to every key process area at every maturity level are categorized by the common features. For example, establishing policies is a common practice in Commitment to Perform and providing training is a common practice in Ability to Perform. Each key process area is described in terms of the key practices that contribute to satisfying its goals and that are allocated to the common features. The key practices describe the specific infrastructure and activities that contribute most to the effective implementation and institutionalization of the key process area. 1999 International Conference on Software Quality Cambridge, MA 3 Table 2. The Key Process Areas in the Software CMM. Level Focus Key Process Areas 5 Optimizing Continuous process improvement Defect Prevention Technology Change Management Process Change Management 4 Managed Product and process quality Quantitative Process Management Software Quality Management 3 Defined Engineering processes and organizational support Organization Process Focus Organization Process Definition Training Program Integrated Software Management Software Product Engineering Intergroup Coordination Peer Reviews 2 Repeatable Project management processes Requirements Management Software Project Planning Software Project Tracking & Oversight Software Subcontract Management Software Quality Assurance Software Configuration Management 1 Initial Competent people and heroics ISO/IEC 15504 -Software Process Assessment ISO/IEC 15504 is intended to harmonize the many different approaches to software process assessment. It has nine parts: Part 1 : Concepts and introductory guide Part 2 : A reference model for processes and process capability Part 3 : Performing an assessment Part 4 : Guide to performing assessments Part 5 : An assessment model and indicator guidance Part 6 : Guide to competency of assessors Part 7 : Guide for use in process improvement Part 8 : Guide for use in determining supplier process capability Part 9 : Vocabulary The reference model in Part 2 documents the set of universal software engineering processes that are fundamental to good software engineering and that cover best practice activities. It describes processes that an organization may perform to acquire, supply, develop, operate, evolve and support software and the process attributes that characterize the capability of those processes. The purpose of the reference model is to provide a common basis for different models and methods for software process assessment, ensuring that results of assessments can be reported in a common context. 1999 International Conference on Software Quality Cambridge, MA 4 The reference model architecture is two dimensional. The process dimension is characterized by process purpose statements, which are the essential measurable objectives of a process. The processes are listed in Table 4. The process capability dimension is characterized by a series of process attributes, applicable to any process, which represent measurable characteristics necessary to manage a process and improve its capability to perform. Each process attribute describes an aspect of the overall capability of managing and improving the effectiveness of a process in achieving its purpose and contributing to the business goals of the organization. There are nine process attributes, which are grouped into capability levels, one at capability level 1 and two each at levels 2-5. Capability levels constitute a rational way of progressing through improvement of the capability of any process. The underlying principles are the same conceptually as the Software CMM maturity levels, although targeted to the process rather than the organization. The six capability levels are described in Table 3. Table 3. The Capability Levels in ISO/IEC 15504-2. Capability Level ISO/IEC 15504-2 Capability Level Description Level 0 Incomplete There is general failure to attain the purpose of the process. There are little or no easily identifiable work products or outputs of the process. Level 1 Performed The purpose of the process is generally achieved. The achievement may not be rigorously planned and tracked. There are identifiable work products for the process, and these testify to the achievement of the purpose. Level 2 Managed The process delivers work products according to specified procedures and is planned and tracked. Work products conform to specified standards and requirements. Level 3 Established The process is performed and managed using a defined process based upon good software engineering principles. Individual implementations of the process use approved, tailored versions of standard, documented processes to achieve the process outcomes. Level 4 Predictable The defined process is performed consistently in practice within defined control limits, to achieve its defined process goals. Level 5 Optimizing Performance of the process is optimized to meet current and future business needs, and the process achieves repeatability in meeting its defined business goals. The process attributes are defined in ISO/IEC 15504-2 and elaborated in ISO/IEC 15504-5 by process indicators, called generic practices in earlier drafts of the evolving standard. Relating ISO/IEC 15504 Processes to CMM Key Process Areas The mapping in Table 4 shows how the topics in ISO/IEC 15504 relate to the equivalent topics in the Software CMM. Topics are typically not isomorphic but are highly correlated. Anyone adequately implementing, for example, the Configuration Management Process in ISO/IEC 15504 could reasonably expect to have satisfied the Software Configuration Management key process area in the Software CMM. Topics are not usually isomorphic because of extensions that may have been added or different levels of abstraction that may have been chosen (e.g., the Development Process in ISO/IEC 1999 International Conference on Software Quality Cambridge, MA 5 12207 addresses the same set of concerns as the Software Product Engineering key process area in the Software CMM). "Subprocesses" and activities in Table 4 are in italics to emphasize that they are components of a larger construct. Where the relationship is indirect, the Software CMM component is in parentheses to highlight the difference in scope. Table 4. Mapping Between ISO/IEC 15504-2 Processes and Software CMM Key Process Areas. ISO/IEC 15504 Processes Software CMM v1.1 CUS.1 Acquisition Software Subcontract Management CUS.1.1 Acquisition preparation Software Subcontract Management, Activity 1 CUS.1.2 Supplier selection Software Subcontract Management, Activity 2 CUS.1.3 Supplier monitoring Software Subcontract Management, Activities 5 and 7-11 CUS.1.4 Customer acceptance Software Subcontract Management, Activity 12 CUS.2 Supply2 (Software Project Planning; Software Project Tracking & Oversight; Software Product Engineering) CUS.3 Requirements elicitation CUS.4 Operation CUS.4.1 Operational use CUS.4.2 Customer support ENG.1 Development Software Product Engineering ENG.1.1 System3 requirements analysis and design ENG.1.2 Software requirements analysis Software Product Engineering, Activity 2 ENG.1.3 Software design Software Product Engineering, Activity 3 ENG.1.4 Software construction Software Product Engineering, Activity 4 ENG.1.5 Software integration Software Product Engineering, Activity 6 2 The Supply Process deals with providing software to the customer that meets the agreed requirements. Establishing a contract, developing the software, and delivering it to the customer, which are the issues for this process, are addressed in various key process areas, although the Supply Process itself is not explicitly specified in the Software CMM. 3 Systems engineering issues, e.g., requirements elicitation, system analysis, and system testing, were not considered within the scope of the Software CMM in version 1.1. 1999 International Conference on Software Quality Cambridge, MA 6 ISO/IEC 15504 Processes Software CMM v1.1 ENG.1.6 Software testing Software Product Engineering, Activity 7 ENG.1.7 System integration and testing (Software Product Engineering, Activities 6 and 7) ENG.2 System and software maintenance SUP.1 Documentation Software Product Engineering, Activity 8 SUP.2 Configuration management Software Configuration Management SUP.3 Quality assurance Software Quality Assurance SUP.4 Verification (Peer Reviews; Software Product Engineering, Activities 5 and 6) SUP.5 Validation Software Product Engineering, Activity 5 SUP.6 Joint review Software Project Tracking & Oversight, Activity 13 SUP.7 Audit (Software Quality Assurance)4 SUP.8 Problem resolution Software Configuration Management, Activity 5 MAN.1 Management5 (Software Project Planning; Software Project Tracking & Oversight; Integrated Software Management) MAN.2 Project management Software Project Planning; Software Project Tracking & Oversight; Integrated Software Management MAN.3 Quality management Software Quality Management MAN.4 Risk management Software Project Planning, Activity 13; Software Project Tracking & Oversight, Activity 10; Integrated Software Management, Activity 10 ORG.1 Organizational alignment6 ORG.4 Infrastructure Organization Process Definition 4 SQA covers both quality assurance and audits. Audits are an independent QA function. The SQA key process area can be implemented as an independent function or not; the requirement is objective verification rather than independent verification. SQA may, or may not, therefore cover the Audit Process in a particular environment. 5 This is a generic planning and management process that is to be applied to any process, rather than project planning specifically. 6 The purpose of the Organizational Alignment Process is to ensure that individuals share a common vision, culture, and understanding of business goals. 1999 International Conference on Software Quality Cambridge, MA 7 ISO/IEC 15504 Processes Software CMM v1.1 ORG.2 Improvement Organization Process Definition ORG.2.1 Process establishment Organization Process Definition ORG.2.2 Process assessment Organization Process Focus, Activity 1 ORG.2.3 Process improvement Organization Process Focus; (Process Change Management) ORG.3 Human resource management Training Program ORG.4 Infrastructure ORG.5 Measurement Measurement and Analysis (common feature)